REMARKS 

This application has been reviewed in light of the Office Action dated 
August 2, 2004. Claims 1-6, 8 5 1 1-29, 31 and 34-54 are presented for examination, of 
which Claims 1 and 22-24 are in independent form. Claims 45-54 have been added to 
assure Applicants of a full measure of protection of the scope to which they deem 
themselves entitled. Claims 1-6, 8, 1 1-15, 17, 18, 22- 29, 31, 34-38, 40 and 41 have been 
amended to define still more clearly what Applicants regard as their invention. Claims 7, 9, 
10, 30, 32 and 33, have been cancelled without prejudice or disclaimer. For the reasons set 
forth in detail below, Applicants respectfully submit that all claims presented for 
examination are in condition for allowance, and favorable reconsideration is requested. 

Initially, a Claim To Priority and a certified copy of the priority document 
for this application were filed on May 14, 2001 (see attached private PAIR print-out). 
Applicants respectfully request acknowledgment of the claim for foreign priority and the 
receipt of the certified copy. 

Also, while the Examiner has initialed and returned a copy of the form PTO- 
1449 filed with one Information Disclosure Statement filed int his application, an earlier 
Information Disclosure Statement and a corresponding Form PTO-1449 were filed on June 
7, 2001 (see PAIR print-out). Applicants therefore respectfully request the Examiner to 
return an initialed copy of that Form PTO-1449 with her next paper. 

Claims 1-18, 20-41, 43 and 44 were rejected under 35 U.S.C. § 103(a) as 
being obvious from U.S. Patent 6,687,878 (Eintracht et al.) in view of U.S. Patent 
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5,706,097 (Schelling et al.). Claims 19 and 42 were rejected under Section 103(a) as being 
obvious from Eintracht in view of Schelling and U.S. Patent 6,272,484 (Martin et al.). 

Independent Claim 1 is directed to a method of annotating an image. The 
method comprises the steps of displaying the image and a plurality of icons, each icon 
being associated with one or more predetermined items of metadata. The method then 
selects at least one of the icons depending on at least one subject rendered at a location 
within the image. The predetermined items of metadata associated with the selected icon 
are linked with a description of the location of the subject within the image. The linked 
metadata and the description are stored as an annotation of the image. 

As described at page 6, lines 1 1-14, of the present specification, a list of 
metadata labels is provided.- The list of metadata labels is preferably provided 
automatically. For example, a list of people's names can be provided automatically by 
extracting the names from an existing database of names, such as an e-mail address book. 
Then as described at page 7, lines 21-25, each label in the list of metadata labels is 
associated with at least one of the generated icons. For example, one of the icons could be 
associated with the name "Jenny Smith". Further as described at page 8, lines 7-12, the 
metadata (e.g., the name "Liza Hayward") associated with the selected icons is stored as an 
association list, in a storage device such as the hard disk drive 210, and linked to the 
rendered image. The position (x,y) and size (width, height) of the bounding box (e.g. 503), 



1/ It is to be understood that the claim scope is not in any way limited by the details of any 
particular portions of the disclosure, or any particular embodiments, referred to. 
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associated with the subject, are also stored in the association list, as in step 113, such that 
the metadata and bounding box information are linked together. 

In making the rejection under Section 103(a), the Examiner concedes that 
Eintracht does not "specifically mention where there is a subject of the annotation is 
displayed with the icon is placed on the images." Applicants concur with this point. 
However, the Examiner contends that Schelling mentions a subject matter descriptor which 
indicates the type of subject matter in the data file {Schelling, col 2, lines 1-15) and that it 
would have been obvious to one of ordinary skill in the art to apply Schelling with 
Eintracht. The Examiner also contends that Eintracht discloses constructing a bounding 
box about locations at which the subject is rendered in the image. 

At col. 2, lines 1-15, Schelling discusses index images including a subject 
matter descriptor, an index code usable for addressing the individual digital data files and a 
type indicator for indicating the type of subject matter in the data file. Examples of index 
images are described at column 2, lines 46-57, of Schelling. In one example, the index 
image (12) includes a subject matter descriptor (13) that is a low resolution version 
(thumbnail) of a still image on the digital recording medium. Schelling states that the index 
image includes an index code (20), located adjacent the thumbnail image for accessing the 
data file containing the still image. In another example, index image (18) includes a subject 
matter descriptor (19) that is a text message describing the data file (i.e., a sound recording 
of a person's voice). 

Eintracht relates to a system for document annotation. At col. 15, lines 
10-23, Eintracht discusses an annotation tool on the screen and placing the cursor in the 
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area of the image where it is desired to place a note. A box is created and the user can enter 
text. Once entered, the text appears on top of the image. A note anchor is created at the 
location the user placed the note. At col. 14, lines 14-22, Eintracht discusses that upon 
receipt of a notes buffer, the client updates a locally stored notes database and then displays 
the contents of the notes buffer in a window within a web browser. If the user has selected 
to view annotations overlying the document (e.g., an image) then the client displays the 
notes, examples of which are shown in Fig. IB of Eintracht. The user then can process the 
notes, create new notes, edit existing ones and/or delete one or more notes. 

Even if a combination of Eintracht and Schelling is assumed to be a proper 
one, the result of such combination would be a system for document annotation, in which 
notes are created by selecting an annotation tool on the screen and placing the cursor in the 
area of the image where it is desired to place a note. A box is then created and the user can 
enter text. Once entered, the notes including the entered text or a thumbnail version of the 
image (see the index of images of Schelling) may appear on top of the image. These notes 
may be used for accessing the data file containing the still image. 

Such combination, however, would not teach, or even expressly/impliedly 
suggest, "displaying the image and a plurality of icons, each icon being associated with 
one or more predetermined items of metadata; selecting at least one of the icons depending 
on at least one subject rendered at a location within the image; linking the predetermined 
items of metadata associated with the selected icon with a description of the location of the 
subject within the image; and storing the linked metadata and the description as an 
annotation of the image", as recited in Claim 1 . 
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These particular features recited in Claim 1 overcome the problems of the 
prior art text entry methods of generating metadata for digital images, as discussed in the 
Background section of the present specification. In such methods, a person sorts through a 
database of digital images, using a computer, and stores with each digital image a short 
textual label (entered by the person) indicating a subject and/or an event depicted by each 
digital image. Such methods require the user to enter in each of the textual labels, which is 
very labor intensive and thus time consuming. 

In fact the system that would be obtained by combining Eintracht and 
Schelling would still suffer from the same problems as the prior art methods discussed in 
the Background section of the present specification, since the combined system would still 
require the user to enter metadata associated with the image for each of the notes or index 
images (see Eintracht at col. 15, lines 16 and 17). Therefore, the combination of Eintracht 
and Schelling, even if a proper one, would actually teach away from the method of Claim 1, 
and in particular away from the recited features of "displaying the image and a plurality of 
icons, each icon being associated with one or more predetermined items of metadata ;.... and 
storing the linked metadata and the description as an annotation of the image". 

Accordingly, Applicant submits that independent Claim 1 is in condition for 

allowance. 

Independent Claims 22-24 are respectively a program claim, computer 
memory medium claim and apparatus claim corresponding to method Claim 1, and the 
arguments presented above are equally applicable to each of them. Accordingly, Claims 
22-24 are each also deemed to be clearly allowable over Eintracht and Schelling. 
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A review of the other art of record has failed to reveal anything which, in 
Applicants' opinion, would remedy the deficiencies of the art discussed above, as references 
against the independent claims herein. Those claims are therefore believed patentable over 
the art of record. 

The other claims in this application are each dependent from one or another 
of the independent claims discussed above and are therefore believed patentable for the 
same reasons. 

For example, Claim 6, dependent from Claim 1 , further recites that a 
bounded region is formed within the image about the location at which the subject is 
rendered in the image, the bounded region being configured to substantially surround the 
subject. For example, as described at page 7, lines 17 to 23 of the present specification, a 
default sized bounding box is presented to the user such that the bounding box substantially 
surrounds a subject. The bounding box is preferably re-sizeable by the user using a mouse 
in a conventional manner. Alternatively, the bounding box can be automatically resized to 
fit the selected subject. 

Applicant submits that none of Eintracht, Schelling or Martin, or any 
possible combination thereof, teach or even expressly/impliedly suggest the particular 
feature of Claim 6 of "forming a bounded region within the image about the location at 
which the subject is rendered in the image, the bounded region being configured to 
substantially surround the subiecf \ 

In contrast, Eintracht merely discloses placing the cursor in the area of the 
image where it desired to place a note and creating a box for the user to enter text. Once 
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entered, the text appears in the form of a "stick" note that appears on top of the image. The 
note anchor is created at the location the user placed the note (see col. 15, lines 10-16). 
Eintracht does not teach or even expressly/impliedly suggest the feature of Claim 6 of 
"forming a bounded region within the image about the location at which the subject is 
rendered in the image, the bounded region being configured to substantially surround the 
subiecf \ 

Accordingly, Applicant submits that dependent Claim 6 also is in condition 
for allowance for at least the reasons stated directly above. For similar reasons, Applicant 
submits that dependent Claim 29, and claims dependent on these claims, are also in 
condition for allowance. 

Since each dependent claim is also deemed to define an additional aspect of 
the invention, however, the individual consideration or reconsideration, as the case may be, 
of the patentability of each on its own merits is respectfully requested. 

In view of the foregoing remarks, is believed that the entire application is in 
condition for allowance, and such action is respectively requested at the Examiner's earliest 
convenience. 
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Applicants' undersigned attorney may be reached in our New York office by 



telephone at (212) 218-2100. All correspondence should continue to be directed to our 
below listed address. 



FITZPATRICK, CELLA, HARPER & SCINTO 
30 Rockefeller Plaza 
New York, New York 10112-3801 
Facsimile: (212)218-2200 

NY_MAIN 403793 vl 



Respectfully submitted, 




Attorney for Applicants 
Leonard P. Diana 
Registration No. 29,296 
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